Signed-off-by: GyuYong Jung <obliviscence@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

The low-level merge machinery has been taught to use CRLF line
termination when inserting conflict markers to merged contents that
are themselves CRLF line-terminated.

* js/xmerge-marker-eol:
  merge-file: ensure that conflict sections match eol style
  merge-file: let conflict markers match end-of-line style of the context

"git worktree" had a broken code that attempted to auto-fix
possible inconsistency that results from end-users moving a
worktree to different places without telling Git (the original
repository needs to maintain backpointers to its worktrees, but
"mv" run by end-users who are not familiar with that fact will
obviously not adjust them), which actually made things worse
when triggered.

* nd/do-not-move-worktree-manually:
  worktree: stop supporting moving worktrees manually
  worktree.c: fix indentation

"git push --force-with-lease" has been taught to report if the push
needed to force (or fast-forwarded).

* aw/push-force-with-lease-reporting:
  push: fix ref status reporting for --force-with-lease

The emulated "yes" command used in our test scripts has been
tweaked not to spend too much time generating unnecessary output
that is not used, to help those who test on Windows where it would
not stop until it fills the pipe buffer due to lack of SIGPIPE.

* js/test-lib-windows-emulated-yes:
  test-lib: limit the output of the yes utility

Signed-off-by: Junio C Hamano <gitster@pobox.com>

It turns out "git clone" over rsync transport has been broken when
the source repository has packed references for a long time, and
nobody noticed nor complained about it.

* jk/drop-rsync-transport:
  transport: drop support for git-over-rsync

Test scripts have been updated to remove assumptions that are not
portable between Git for POSIX and Git for Windows, or to skip ones
with expectations that are not satisfiable on Git for Windows.

* js/mingw-tests: (21 commits)
  gitignore: ignore generated test-fake-ssh executable
  mingw: do not bother to test funny file names
  mingw: skip a test in t9130 that cannot pass on Windows
  mingw: handle the missing POSIXPERM prereq in t9124
  mingw: avoid illegal filename in t9118
  mingw: mark t9100's test cases with appropriate prereqs
  t0008: avoid absolute path
  mingw: work around pwd issues in the tests
  mingw: fix t9700's assumption about directory separators
  mingw: skip test in t1508 that fails due to path conversion
  tests: turn off git-daemon tests if FIFOs are not available
  mingw: disable mkfifo-based tests
  mingw: accomodate t0060-path-utils for MSYS2
  mingw: fix t5601-clone.sh
  mingw: let lstat() fail with errno == ENOTDIR when appropriate
  mingw: try to delete target directory before renaming
  mingw: prepare the TMPDIR environment variable for shell scripts
  mingw: factor out Windows specific environment setup
  Git.pm: stop assuming that absolute paths start with a slash
  mingw: do not trust MSYS2's MinGW gettext.sh
  ...

An earlier adjustment of test mistakenly used write_script
to prepare a file whose exact content matters for the test;
reverting that part fixes the breakage for those who use
SHELL_PATH that is different from /bin/sh.

* mg/mingw-test-fix:
  t9100: fix breakage when SHELL_PATH is not /bin/sh

The automatic typo correction applied to an alias was broken
with a recent change already in 'master'.

* nd/clear-gitenv-upon-use-of-alias:
  restore_env(): free the saved environment variable once we are done
  git: simplify environment save/restore logic
  git: protect against unbalanced calls to {save,restore}_env()
  git: remove an early return from save_env_before_alias()

The "user.useConfigOnly" configuration variable can be used to
force the user to always set user.email & user.name configuration
variables, serving as a reminder for those who work on multiple
projects and do not want to put these in their $HOME/.gitconfig.

* da/user-useconfigonly:
  ident: add user.useConfigOnly boolean for when ident shouldn't be guessed
  fmt_ident: refactor strictness checks

Some calls to strcpy(3) triggers a false warning from static
analysers that are less intelligent than humans, and reducing the
number of these false hits helps us notice real issues.  A few
calls to strcpy(3) in test-path-utils that are already safe has
been rewritten to avoid false wanings.

* jk/test-path-utils-xsnprintf:
  test-path-utils: use xsnprintf in favor of strcpy

Some calls to strcpy(3) triggers a false warning from static
analysers that are less intelligent than humans, and reducing the
number of these false hits helps us notice real issues.  A few
calls to strcpy(3) in "git rerere" that are already safe has been
rewritten to avoid false wanings.

* jk/rerere-xsnprintf:
  rerere: replace strcpy with xsnprintf

The documentation for "git clean" has been corrected; it mentioned
that .git/modules/* are removed by giving two "-f", which has never
been the case.

* mm/clean-doc-fix:
  Documentation/git-clean.txt: don't mention deletion of .git/modules/*

* ah/stripspace-optstring:
  stripspace: call U+0020 a "space" instead of a "blank"

The vimdiff backend for "git mergetool" has been tweaked to arrange
and number buffers in the order that would match the expectation of
majority of people who read left to right, then top down and assign
buffers 1 2 3 4 "mentally" to local base remote merge windows based
on that order.

* dw/mergetool-vim-window-shuffle:
  mergetool: reorder vim/gvim buffers in three-way diffs

Add a ref backend method for do_for_each_ref.

Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Now that we have moved do_for_each_ref into refs.c, it no longer needs
to be public.

Signed-off-by: Ramsay Jones <ramsay@ramsayjones.plus.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Alternate refs backends might still use files to store per-worktree
refs.  So the files backend's ref-loading infrastructure should be
available to those backends, just for use on per-worktree refs.  Add
do_for_each_per_worktree_ref, which iterates over per-worktree refs.

Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

In the file-based backend, the reflog piggybacks on the ref lock.
Since other backends won't have the same sort of ref lock, ref backends
must also handle reflogs.

Signed-off-by: Ronnie Sahlberg <rsahlberg@google.com>
Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Signed-off-by: Ronnie Sahlberg <rsahlberg@google.com>
Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

In the file-based backend, delete_refs has some special optimization
to deal with packed refs.  In other backends, we might be able to make
ref deletion faster by putting all deletions into a single
transaction.  So we need a special backend function for this.

Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Alternate refs backends might not need the refs/heads directory and so
on, so we make ref db initialization part of the backend.

Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Forbid cross-backend ref renames.  This would be pretty weird, but
since it will break, we should prevent it.

Also make the files backend deal with all non-normal ref renames.

Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Instead of using a files-backend-specific struct ref_lock, the generic
ref_transaction struct should provide a void pointer that backends can use
for their own lock data.

Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

The check for duplicate refnames in a transaction is needed for
all backends, so move it to the common code.

ref_transaction_commit_fn gains a new argument, the sorted
string_list of affected refnames.

Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

The refs infrastructure learns about log-only ref updates, which only
update the reflog.  Later, we will use this to separate symbolic
reference resolution from ref updating.

Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

When renaming refs, don't dereference either the origin or the destination
before renaming.

The origin does not need to be dereferenced because it is presently
forbidden to rename symbolic refs.

Not dereferencing the destination fixes a bug where renaming on top of
a broken symref would use the pointed-to ref name for the moved
reflog.

Add a test for the reflog bug.

Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

When locking a symbolic ref to expire a reflog, lock the symbolic
ref (using REF_NODEREF) instead of its referent.

Add a test for this.

Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Before committing ref updates, split symbolic ref updates into two
parts: an update to the underlying ref, and a log-only update to the
symbolic ref.  This ensures that both references are locked correctly
while their reflogs are updated.

It is still possible to confuse git by concurrent updates, since the
splitting of symbolic refs does not happen under lock. So a symbolic ref
could be replaced by a plain ref in the middle of this operation, which
would lead to reflog discontinuities and missed old-ref checks.

All callers to lock_ref_sha1_basic outside of ref_transaction_commit
now set REF_NODEREF.  And ref_transaction_commit only happens on
already-derefernced refs.  So we can assume REF_NODEREF when resolving
inside this function.

Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Always handle non-normal (per-worktree or pseudo) refs in the files
backend instead of alternate backends.

Sometimes a ref transaction will update both a per-worktree ref and a
normal ref.  For instance, an ordinary commit might update
refs/heads/master and HEAD (or at least HEAD's reflog).

Updates to normal refs continue to go through the chosen backend.

Updates to non-normal refs are moved to a separate files backend
transaction.

Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

All submodules must have the same ref storage (for now).  Confirm that
this is so before attempting to do anything with submodule refs.

Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Add a new option, --ref-storage, to allow the ref storage backend to
be set on new clones.

Submodules must use the same ref storage as the parent repository, so
we also pass the --ref-storage option option when cloning
submodules.

Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: SZEDER Gábor <szeder@ira.uka.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

git svn learns to pass the ref-storage command-line argument (to init
and clone) through to git init.

Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: SZEDER Gábor <szeder@ira.uka.de>
Signed-off-by: Eric Wong <normalperson@yhbt.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Add new function register_ref_storage_backends().  This new function
registers all known ref storage backends... once there are any other
than the default.  For now, it just registers the files backend.

Call the function before calling set_ref_storage_backend.

Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

This sets up the existing backend early, so that other code which
reads refs is reading from the right place.

Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

It will soon be useful for resolve_ref_unsafe to support submodules.
But since it is called from so many places, changing it would have
been painful.  So a new resolve_ref_unsafe_submodule is created
instead.

read_raw_ref learns a submodule argument to support this use case.

The files backend doesn't need this functionality, but it doesn't
hurt.

Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Add a database backend for refs using LMDB.  This backend runs git
for-each-ref about 30% faster than the files backend with fully-packed
refs on a repo with ~120k refs.  It's also about 4x faster than using
fully-unpacked refs.  In addition, and perhaps more importantly, it
avoids case-conflict issues on OS X.

LMDB has a few features that make it suitable for usage in git:

1. It is licensed under the OpenLDAP Public License, which is
GPL-compatible [1].

[1] http://www.gnu.org/licenses/license-list.en.html#newOpenLDAP

2. It is relatively lightweight; it requires only one header file, and
the library code takes under 64k at runtime.

3. It is well-tested: it's been used in OpenLDAP for years.

4. It's very fast.  LMDB's benchmarks show that it is among
the fastest key-value stores.

5. It has a relatively simple concurrency story; readers don't
block writers and writers don't block readers.

Ronnie Sahlberg's original version of this patchset used tdb.  The
major disadvantage of tdb is that tdb is hard to build on OS X.  It's
also not in homebrew.  So lmdb seemed simpler.

To test this backend's correctness, I added support to the test suite
to run under LMDB.  Dozens of tests use manual ref/reflog
reading/writing, or create submodules without passing --ref-storage to
git init.  If those tests are changed to use the update-ref machinery
or test-refs-lmdb-backend (or, in the case of packed-refs, corrupt
refs, and dumb fetch tests, are skipped), the only remaining failing
tests are the git-new-workdir tests.  Later, we'll add this support
to the test suite.

Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Add tests for the database backend.

Signed-off-by: David Turner <dturner@twopensource.com>
Helped-by: Dennis Kaarsemaker <dennis@kaarsemaker.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Add a --ref-storage argument to make it easy to test alternate ref
storage backends.  This means that every test that calls git init or
git clone must use the new ref storage argument.

Modify many tests to work under alternate ref storage backends.

Introduce abstractions for raw ref/reflog reading/writing in tests
instead of directly frobbing the filesystem.

Conditionally skip tests that are not expected to succeed under this
condition. Most of this is straightforward. Of particular note are the
following test changes:

* The rearrangement of commands in t1401 is because without HEAD in
the right place, git doesn't recognized the trash dir as a git repo,
so no git commands work.

* t1430-bad-ref-name specifically blocks lmdb because other alternate
backends might want to keep this test.

* Some tests contain comments explaining their skippage.

Signed-off-by: David Turner <dturner@twopensource.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>